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A local search algorithm solving an NP-complete optimisation problem can be viewed as 
a stochastic process moving in an 'energy landscape' towards eventually finding an optimal 
solution. For the random 3-satisfiability problem, the heuristic of focusing the local moves on 
the presently unsatisfied clauses is known to be very effective: the time to solution has been 
observed to grow only linearly in the number of variables, for a given clauses-to- variables ratio 
a sufficiently far below the critical satisfiability threshold a c w 4.27. We present numerical 
results on the behaviour of three focused local search algorithms for this problem, considering 
in particular the characteristics of a focused variant of the simple Metropolis dynamics. We 
estimate the optimal value for the "temperature" parameter i] for this algorithm, such that 
its linear-time regime extends as close to a c as possible. Similar parameter optimisation is 
performed also for the well-known WalkSAT algorithm and for the less studied, but very well 
performing Focused Record-to-Record Travel method. We observe that with an appropriate 
choice of parameters, the linear time regime for each of these algorithms seems to extend 
well into ratios a > 4.2 — much further than has so far been generally assumed. We discuss 
the statistics of solution times for the algorithms, relate their performance to the process 
of "whitening" , and present some conjectures on the shape of their computational phase 
diagrams. 



I. INTRODUCTION 

The 3-satisfiability (3-S AT) problem is one of the paradigmatic NP-complete problems |8] . An 
instance of the problem is a formula consisting of M clauses, each of which is a set of three literals, 
i.e. Boolean variables or their negations. The goal is to find a solution consisting of a satisfying 
truth assignment to the N variables, such that each clause contains at least one literal evaluating 
to 'true', provided such an assignment exists. In a random 3-SAT instance, the literals comprising 
each clause are chosen uniformly at random. 

It was observed in that random 3-SAT instances change from being generically satisfiable 
to being generically unsatisfiable when the clauses-to-variables ratio a = M/N exceeds a critical 
threshold a c . Current numerical estimates P| suggest that this satisfiability threshold is located 
approximately at a c ~ 4.267. For a general introduction to aspects of the satisfiability problem 
seefl. 

Two questions are often asked in this context: how to solve random 3-SAT instances effectively 
close to the threshold a c , and what can be said of the point at which various types of algorithms 

fl 15] 



begin to falter. Recently progress has been made by the "survey propagation" method 
that essentially iterates a guess about the state of each variable, in the course of fixing an ever 
larger proportion of the variables to their "best guess" value. 

In this article we investigate the performance of algorithms that try to find a solution by local 
moves, flipping the value of one variable at a time. When the variables to be flipped are chosen 
only from the unsatisfied (unsat) clauses, a local search algorithm is called focusing. A particularly 



23J, which makes 



well-known example of a focused local search algorithm for 3-SAT is WalkSAT 
"greedy" and random flips with a fixed probability. Many variants of this and other local search 
algorithms with different heuristics have been developed; for a general overview of the techniques 
see U. 

n 

Our main emphasis will be on a focused variant of the standard Metropolis dynamics [131 ] of 
spin systems, which is also the base for the well-known simulated annealing optimization method. 
When applied to the 3-SAT problem, we call this dynamics the Focused Metropolis Search (FMS) 
algorithm. We also consider the Focused Record-to-Record Travel (FRRT) algorithm [l, 22], which 
imposes a history-dependent upper limit for the maximum energy allowed during its execution. 

The space of solutions to 3-SAT instances slightly below a c is known to develop structure. 
Physics-based methods from mean-field -like spin glasses imply through replica methods that the 



solutions become "clustered", with a threshold of a ~ 3.92 15]. The clustering implies that 3 
measured e.g. by Hamming distance the different solutions in the same cluster are close to each 
other. A possible consequence of this is the existence of cluster "backbones", which means that 
in a cluster a certain fraction of the variables is fixed, while the others can be varied subject to 
some constraints. The stability of the replica ansatz becomes crucial for higher a, and it has been 
suggested that this might become important at a ~ 4.15 17]. The energy landscape in which the 
local algorithms move is however a finite-temperature version. The question is how close to a c one 
can get by moving around by local moves (spin flips) and focusing on the unsat clauses. 

Our results concern the optimality of this strategy for various algorithms and a. First, we 
demonstrate that WalkSAT works in the "critical" region up to a > 4.2 with the optimal noise 
parameter p ~ 0.57. Then, we concentrate on the numerical performance of the FMS method. In 
this case, the solution time is found to be linear in N within a parameter window r\ m i n < rj < rj max , 
where r\ is essentially the Metropolis temperature parameter. More precisely, we observe that within 
this window, the median and all other quantiles of the solution times normalised by N seem to 
approach a constant value as N increases. A stronger condition would be that the distributions of 
the solution times be concentrated in the sense that the ratio of the standard deviation and the 
average solution time decreases with increasing N. While numerical studies of the distributions 
do not indicate heavy tails that would contradict this condition, we can of course not at present 
prove that this is rigorously true. The existence of the rj window implies that for too large rj the 
algorithm becomes entropic, while for too small rj it is too greedy, leading to freezing of degrees of 
freedom (variables) in that case. 

The optimal rj = rj op t(a), i.e. that rj for which the solution time median is lowest, seems to 
increase with increasing a. We have tried to extrapolate this behaviour towards a c and consider it 
possible that the algorithm works, in the median sense, linearly in N all the way up to a c . This 
is in contradiction with some earlier conjectures. All in all we postulate a phase diagram for the 
FMS algorithm based on two phase boundaries following from too little or too much greed. Of this 
picture, one particular phase space point has been considered before P, 13], since it happens to be 
the case that for ri = 1 the FMS coincides the so called Random Walk method llal, which is known 
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to suffer from metastability at a « 2.7 |3j, 25]. 

The structure of this paper is as follows: In Section 2 we present as a comparison the WalkSAT 
algorithm, list some of the known results concerning the random 3-SAT problem, and test the 
WalkSAT for varying p and a values. In Section 3 we go through the FMS algorithm in detail, and 



WalkSAT(F.p) : 4 
s = random initial truth assignment; 
while flips < max_flips do 

if s satisfies F then output s & halt, else: 

- pick a random unsatisfied clause C in F; 

- if some variables in C can be flipped without 
breaking any presently satisfied clauses, 
then pick one such variable x at random; else: 

- with probability p, pick a variable x 
in C at random; 

- with probability (1-p) , pick some x in C 
that breaks a minimal number of presently 
satisfied clauses; 

- flip x. 

FIG. 1: The WalkSAT algorithm. 

present extensive numerical simulations with it. Section 4 presents as a further comparison similar 
data for the FRRT algorithm. In Section 5 we discuss how the notion of whitening is related 
to the behaviour of FMS with low rj. Finally, section 6 finishes the paper with a summary. 



II. LOCAL SEARCH FOR SATISFIABILITY 



It is natural to view the satisfiability problem as a combinatorial optimisation task, where the 
goal for a given formula F is to minimise the objective function E = Ep(s) = the number of 
unsatisfied clauses in formula F under truth assignment s. The use of local search heuristics in this 



context was promoted e.g. by Selman et al. in 
E can be taken to be the energ y of the system 
Selman et al. introduced in 



24| and by Gu in |9|. Viewed as a spin glass model, 



2J] the simple greedy GSAT algorithm, whereby at each step the 



variable to be flipped is determined by which flip yields the largest decrease in E, or if no decrease 
is possible, then smallest increase. This method was improved in by augmenting the simple 
greedy steps with an adjustable fraction p of pure random walk moves, leading to the algorithm 
NoisyGSAT. 

In a different line of work, Papadimitriou introduced the very important idea of focusing 
the search to consider at each step only those variables that appear in the presently unsatisfied 



clauses. Applying this modification to the NoisyGSAT method leads to the WalkSAT [23j algorithm" 
(Figure which is the currently most popular local search method for satisfiability. 

In |23j, Selman et al. presented some comparisons among their new NoisyGSAT and WalkSAT 
algorithms, together with some other methods. These experiments were based on a somewhat 
unsystematic set of benchmark formulas with N < 2000 at a ~ a c , but nevertheless illustrated the 
significance of the focusing idea, since in the results reported, WalkSAT outperformed NoisyGSAT 
by several orders of magnitude. 

More recently, Barthel et al. ■<! performed systematic numerical experiments with Papadim- 
itriou's original Random Walk method at N = 50, 000, a = 2.0 . . . 4.0. They also gave an analytical 
expiation for a _ ,„ the d y na mira of this a^itta at o dyn „ 2.7, a lr ead y o bs e™d by 
Parkes |2JJ] . When a < adym satisfying assignments are generically found in time that is linear 
in the number of variables, whereas when a > «H yr , exponential time is required. (Similar results 



were obtained by Semerjian and Monasson in 



25J, though with smaller experiments (N = 500).) 



The reason for this dynamical threshold phenomenon seems to be that at a > adyn the search 
equilibrates at a nonzero energy level, and can only escape to a ground state through a large 



enough random fluctuation. A rate-equation analysis of the method 
approximation of ad yn ~ 2.71. 



yields a very well matching 



28] See also |26j ] for further analyses of the Random Walk method 



on random 3-SAT. 

WalkSAT is more powerful than the simple Random Walk, because in it focusing is accompanied 
by other heuristics. However, it is known that the behaviour of WalkSAT is quite sensitive to the 
choice of the noise parameter p. E.g. Parkes experimented with the algorithm using a noise 
value p = 0.3 and concluded that with this setting the algorithm works in linear time at least up 
to a = 3.8. Even this result is not the best possible, since it has been estimated 10. 11. 2^| that 
for random 3-SAT close to the satisfiability transition the optimal noise setting for WalkSAT is 
p ~ 0.55. (Actually our numerical experiments, reported below, suggest that the ideal value is 
closer to p ~ 0.57.) 

n 

These positive results notwithstanding, it has been generally conjectured (e.g. in jj|) that no 
local search algorithm can work in linear time beyond the clustering transition at ay ~ 3.92. In 
a series of recent experiments, however, Aurell et al. |2J concluded that with a proper choice of 
parameters, the median solution time of WalkSAT remains linear in N up to at least a = 4.15, the 
onset of 1-RSB symmetry breaking. Our experiments, reported below, indicate that the median 



time in fact remains linear even beyond that, in line with our previous results 
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WalkSAT (N = 10"5) 



WalkSAT (N = 10"5) 
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(a) Complete data 



(b) Medians and quartiles 



FIG. 2: Normalised solution times for WalkSAT, a = 3.8. . .4.3. 



23 ] on randomly generated 



Figure [21 illustrates our experiments with the WalkSAT algorithm, 
formulas of size N = 10 5 , various values of the noise parameter p, and values of a starting from 
3.8 and increasing at increments of 0.2 up to 4.22. For each (p, a)-combination, 21 formulas were 
generated, and for each of these the algorithm was run until either a satisfying solution was found 
or a time limit of 80000 x N flips was exceeded. Figure |2^a) shows the solution times t so i, measured 
in number of flips normalised by N, for each generated formula. Figure IHb) gives the medians and 
quartiles of the data for each value of a. 

As can be seen from the figures, for value p = 0.45 of the noise parameter WalkSAT finds satis- 
fying assignments in roughly time linear in N, with the coefficient of linearity increasing gradually 
with increasing a, up to approximately a = 4.1 beyond which the distribution of solution times for 
the algorithm diverges. For p = 0.55, this linear regime extends further, up to at least a = 4.18, 
but for p = 0.65 it seems to end already before a = 3.9. For the best value of p we have been 
able to experimentally determine, p = 0.57, the linear regime seems to extend up to even beyond 
a = 4.2. 

To investigate the convergence of the solution time distributions, we tested the WalkSAT algo- 
rithm with p = 0.55 at a = 4.15 and a = 4.20, in both cases with randomly generated sets of 100 
formulas of sizes N = 10 4 ,3 x 10 4 ,10 5 ,3 x 10 5 and 10 6 . Fi gure El shows the cumulative distributions 
of the solution times normalised by achieved in these tests. As can be seen, for a = 4.15 the 
distributions are well-defined, with normalised medians and all other quantiles converging to a fi- 
nite value for increasing N. However, for a = 4.20, the distributions seem to diverge, with median 



WalkSAT (p = 0.55, alpha = 4.1 5) 



WalkSAT (p = 0.55, alpha = 4.20) 





(a) a = 4.15 



(b) a = 4.20 



FIG. 3: Cumulative solution time distributions for WalkSAT with p = 0.55. 



WalkSAT (N = 10*5) 
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FIG. 4: Normalised solution times for WalkSAT with a = 4.10 . . . 4.22, p = 0.50 . . . 0.60. 

values increasing with increasing N. [3^ 

In order to estimate the optimal value of the WalkSAT noise parameter, i.e. that value of p for 
which the linear time regime extends to the biggest value of a, we generated test sets consisting 
of 21 formulas, each of size N = 10 5 , for a values ranging from 4.10 to 4.22 at increments of 
0.02, and for each a for p values ranging from 0.50 to 0.60 at increments of 0.01. WalkSAT was 
then run on each resulting (a,p) test set; the medians and quartiles of the observed solution time 
distributions are shown in Figure The data suggest that the optimal value of the noise parameter 
is approximately p = 0.57. Figure [3 shows the empirical solution time distributions at a = 4.20 
for p = 0.57. In contrast to Figure |2tb), now the quantiles seem again to converge to a finite value 
for increasing N, albeit more slowly than in the case of the simpler a = 4.15 formulas presented in 
Figure Efa) . 



WalkSAT (p = 0.57, alpha = 4.20) 
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FIG. 5: Cumulative solution time distributions for WalkSAT with p — 0.57, a = 4.20. 

FMS(F,eta) : 

s = random initial truth assignment; 
while flips < max_flips do 

if s satisfies F then output s & halt, else: 
pick a random unsatisfied clause C in F; 
pick a variable x in C at random; 
let x' = flip(x), s' = s[x'/x]; 
if E(s') <= E(s) then flip x, else: 
flip x with prob. eta~ (E(s ' ) -E(s) ) . 

FIG. 6: The Focused Metropolis Search algorithm. 



III. FOCUSED METROPOLIS SEARCH 



From an analytical point of view, the WalkSAT algorithm is rather complicated, with its in- 
terleaved greedy and randomised moves. Thus, it is of interest to investigate the behaviour of the 
simpler algorithm obtained by imposing the focusing heuristic on a basic Metropolis dynamics [t3 |. 

We call the resulting algorithm, outlined in Figure H3 the Focused Metropolis Search (FMS) 
method. The algorithm is parameterised by a number rj, < rj < 1, which determines the probabil- 
ity of accepting a candidate variable flip that would lead to a unit increase in the objective function 
E. (Thus in customary Metropolis dynamics terms, r\ = e x l T , where T is the chosen computational 
temperature. Note, however, that detailed balance does not hold with focusing.) 

We repeated the data collection procedure of Figure |2] for the FMS algorithm with various 
parameter values. The results for rj = 0.2, 0.3, 0.4 and rj = 0.36 (the best value we were able to 
find) are shown in Figured note that also rejected flips are here included in the flip counts. As can 
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(a) Complete data 



(b) Medians and quartiles 



FIG. 7: Normalised solution times for FMS, a = 3.8 . . .4.3. 



Focused Metropolis Search (eta = 0.3, alpha = 4.00) 



Focused Metropolis Search (eta = 0.3, alpha = 4.10) 
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(a) a = 4.00 



(b) a = 4.10 



FIG. 8: Cumulative solution time distributions for FMS with rj = 0.3. 



be seen, the behaviour of the algorithm is qualitatively quite similar to WalkSAT. For parameter 
value r] = 0.2, the linear time regime seems to extend up to roughly a = 4.06, for r] = 0.3 up to 
at least a = 4.14, and for r/ = 0.36 even beyond a = 4.20; however for r] = 0.4 again only up to 
maybe a = 4.08. To test the convergence of distributions, we determined the empirical cumulative 
distributions of FMS solution times for rj = 0.3 at a = 4.0 and a = 4.1, in a similar manner as in 
Figure |31 The results are shown in Figure |5J 

To determine the optimal value of the ij parameter we proceeded as in Figure HJ mapping out 
systematically the solution time distributions of the FMS algorithm for a increasing from 4.10 to 
4.22 and r\ ranging from 0.28 to 0.38. The results, shown in Figure El suggest that the optimal 
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FIG. 9: Normalised solution times for FMS with 77 = 0.28 . . . 0.38, a = 4.10 . . . 4.22. 
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FIG. 10: Cumulative solution time distributions for FMS with rj = 0.36, a = 4.20. 

value of the parameter is approximately rj = 0.36. 

In order investigate the algorithm's behaviour at this extreme of its parameter range, we deter- 
mined the empirical cumulative distributions of the FMS solution times for rj = 0.36 at a = 4.20. 
The results, shown in Figure ITU1 suggest that even for this high value of a, the FMS solution times 
are linear in N, with all quantiles converging to a finite value as N increases. 

The linearity of FMS can fail due to the formation of heavy tails. This, with a given a and a 
not too optimal, large 77 would imply that the solution time t so i has at least a divergent mean (first 
moment) and perhaps a divergent median as well. This can be deliberated upon by considering the 
"scaling ansatz" 



Pitsoi) ~ (t sol )- a f(t sol /N b ) 



(1) 



where f(x) = const for x small, and / — > rapidly for x > 1. This simply states that for a fixed 
./V there has to be a maximal solution time (even exponentially rare) since the FMS is "ergodic" 
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FIG. 11: (a) The TV-dependence of the average solution time for a = 4.1 and -q = 0.3. (b) The difference of 
the upper and lower quartiles in t so i vs. TV for the same parameters. The line shows l/yiV- behaviour as a 
guide for the eye. 



or able to get out of local minima. 



3l| The condition that (t soi ) ~ N b( - 2 ~ a ^ be divergent with 
N would then set a relation for the exponents a and b. Our experiments in the linear regime 
have not yielded any evidence for a distribution following such an ansatz, and moreover we have 
not explored systematically the non-linear parameter region, where such behaviour might make 
considering scalings as Eq. (^Q) interesting. The average solution time, in terms of flips per spin, 
is shown in Figure ITTT a') for a = 4.1. Together with Figure ITTT b). showing the tendency for the 
width of the distribution to diminish as 1/yN, this 1/iV-behaviour implies rather trivial finite 
size corrections to P(t so i). In the Figure we depict the width of the distribution P measured by 
quantiles instead of the standard deviation, since this is the most sensible measure given the nature 
of the data. 

We also tried to extract the best possible performance of the algorithm as a function of a. 
Using the data for varying rj allows one to extract roughly the optimal values rj op t(a) which are 
demonstrated in Figure lT2l As can be seen, the data indicate a roughly linear increase of the optimal 
77 with in particular no notice of the approach to a c or to an algorithm-dependent maximum value 
otmax- The same data can be also utilised to plot, for a fixed N (recall the FMS runs linearly 
in this regime) the solution time at the optimal noise parameter rj. Figure shows that this as 
expected diverges. Attempts to extract the value a max limiting the linear regime by fitting to 
various divergences of the kind t so i ~ (a max — a)~ b do not allow one to make a statistically reliable 
conclusion as to whether a max < a c , though . The reason for this is, as far as the data is concerned, 
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FIG. 12: Optimal 77 vs. a, for N = 10 5 
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FIG. 13: Solution time at r) opt vs. a. 
the distance of the a studied to any plausible value of a max . See Figure EJ 



IV. FOCUSED RECORD-TO-RECORD TRAVEL 



Record-to-Record Travel, a very simple stochastic local optimisation algorithm outlined in Fig- 

_ fl 
ure!15( was introduced by Dueck in |2|. It restricts acceptable moves based on the best configuration 

found so far, and is in this way non-Markovian. Dueck claimed good results on solving 442-city 

and 532-city instances of the Travelling Salesman Problem, but after that the algorithm has been 
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FIG. 14: Solution time t so i at 77 = r] opt (a) for N = 10 5 : possible divergence. 

RRT(E.d) : 

s = random initial configuration; 

s* = s ; E* = E(s) ; 

while moves < max_moves do 

if s is a global min. of E then output s & halt, 

else : 

pick a random neighbour s' of s; 
if E(s') <= E* + d then let s = s'; 
if E(s') < E* then: 
s* = s' ; E* = E(s') . 

FIG. 15: The Record-to-Record Travel algorithm. 

little used. Initial experiments on applying the focused version of this algorithm to the satisfiability 
problems were reported in jzj ]. 

In applying Record-to-Record Travel to satisfiability, one again chooses as the objective function 
E(s) = number of clauses unsatisfied by truth assignment s. The local search is performed over 
single- variable flip neighbourhoods. We also impose the focusing condition requiring that the flipped 
variables are always chosen from unsatisfied clauses. (Precisely: one unsatisfied clause is chosen at 
random, and from there a variable at random.) This leads to the focused RRT (FRRT) algorithm. 

The algorithm is surprisingly powerful, as indicated by the results in Figure ED Interestingly, at 
least within the range of values tested, the FRRT algorithm does not seem to have an optimal value 
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FIG. 16: Normalised solution times for FRRT, a = 3.8 . . .4.3. 
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FIG. 17: Cumulative solution time distributions for FRRT with d = 9. 



for the deviation parameter d, but larger values of d appear to always extend the linear regime 
of the algorithm further, albeit with increasing coefficients of linearity. E.g. for d = 5 the linear 
regime seems to extend only to about a = 4.12, but for d = 9 already to a = 4.2, and for d = 11 
even further. 

To validate these results, we again determined the normalised cumulative solution time distri- 
butions for the FRRT algorithm with d = 9 at both a = 4.15 and a = 4.20, similarly as in FigureEl 
As can be seen from Figure IT71 FRRT behaves for these parameter values more predictably than 
WalkSAT at p = 0.55, and comparably to WalkSAT at the near-optimal noise parameter value 
of p = 0.57. (Interestingly, the FRRT normalised median solution times seem to decrease slightly 



15 



Focused Record-to- Record Travel (alpha = 4.15) 




100 1000 
flips/N 



N = 10M - 
N = 3*10"4 

N = 10"5 - 
N = 3*1 0"5 - 

N = 1 0"6 ■ 



10000 



Focused Record -to- Record Travel (alpha = 4.20) 
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FIG. 18: Cumulative solution time distributions for FRRT with d = 7. 

with increasing N. This phenomenon is undoubtedly connected to the smoothing of the random 
3-SAT energy landscapes for increasing N, but otherwise we have no clear explanation for it. It 
can also be observed that the shapes of the cumulative distributions are qualitatively different from 
the FMS case.) 

For comparison, Figure IT8l illustrates the corresponding distributions for d = 7. As can be seen, 
for a = 4.15 FRRT converges even with d = 7, but for a = 4.20 a larger value of d is required. 

V. WHITENING AND THE LINEAR TIME REGIME 



As already mentioned in Section 2, WalkSAT finds a solution in linear time when a < adyn(p)) 
where «dyn is a dynamical threshold depending on the noise parameter p. When a > a^yn, the 
energy falls to a metastable level around which it remains fluctuating. In this metastable state new 
unsat clauses are produced at approximately the same rate as they are being eliminated, and a 
solution is found only by a lucky fluctuation, for which the waiting time is exponential in iV ill^. 

Our experiments show this kind of behaviour also for FMS.|22j When the parameter r\ is de- 
creased away from one, the corresponding a^ yn (r]) value increases (cf. Figure|^]in Section 6). Since 
the rate of production of unsatisfied clauses vanishes as rj decreases toward zero, it is tempting to 
think that if FMS is in the linear time regime for some particular values of a and 77, then it remains 
so also for all smaller rj given that particular a. But this seems not always to be the case. 

It is a common observation that excessive greediness can be detrimental to the quality of op- 
timisation results. How is such behaviour exhibited in the case of the FMS algorithm on random 



WHITENING (F , s) : 16 
mark all the clauses white (*, joker, wild card) except those that 

have only one true literal (thus, also unsatisfied clauses are marked 
white, so that they do not affect the whitening process); 
loop 

- mark those variables white that appear as satisfying literals 
only in white clauses; 

- halt, if all the variables are white (s is completely white); 

- halt, if no new variables became white in this round (s has a core); 

- mark those clauses white that contain at least one white variable. 

FIG. 19: The whitening algorithm for a truth assignment s. 

3-SAT? We think that the answer is related to the concept of whitening, introduced in the context 
of the graph colouring problem (hence the name) in ,13] and applied to random 3-SAT in 
The constraints between variables contain slack that can be consumed too quickly by a greedy 
search. This slack is due to "safe" clauses, i.e. clauses that contain at least two true literals. For 
some a values, if r/ is too low, FMS typically drives the system to a state where some unsatisfied 
clauses can no longer be eliminated "locally" .|33j] 

The whitening alaorithm^\ presented in Figure IT^l searches for frozen variables in a given truth 
assignment. Flipping any frozen variable produces at least one new unsat clause that contains 
only frozen variables. To get rid of that unsat clause, one of its variables must be flipped, and 
that produces again at least one new unsat clause with all of its other variables frozen, and so on. 
The only way to do away with all of these newly produced unsat clauses is by loops, i.e. by flip 
sequences that encounter some variables more than once. In random 3-SAT instances these loops 
are typically long, growing as 0(log N). So when N —* oo, also the loops become infinitely long. If 
there are no frozen variables, the truth assignment is said to be completely white, otherwise it has 
a core of frozen variables (the term core in this connection is from |12|). 

When energy decreases during a search, a transition can occur from a state where the encoun- 
tered truth assignments are generically completely white to a state where the truth assignments 
generically have a core of frozen variables. We call this the freezing transition. It is natural to 
assume that when a freezing transition has happened, the remaining unsatisfied clauses cannot be 
eliminated in linear time, because then the search needs loops growing with the system size. 

In Figure I2U1 we present estimates of the freezing transition energy density et = Ef/N for a few 



FIG. 20: Freezing transition energies et, when using FMS. Each point corresponds to a single run (N = 10 5 ). 
Dotted lines arc linear fits. 

sample runs of the FMS algorithm. We have not studied the dependence of Ef on N, but it appears 
to be self-averaging, so that for any a and 77 for which a freezing transition occurs, e/ converges to 
a constant value as N —* 00. Hence the et values given in Figures [201 and |^ are similar to their 
N — > 00 limits. Also the freezing transition time tt (measured in flips/TV) seems to converge for 
many (a, 7?)-pairs, but it is not at all clear that this needs to be so in every case. There may exist 
some region of a and rj values for which a superlinear number of flips are required before either a 
satisfying solution or a truth assignment with a core is reached (cf. Figure 12*2*)) . 

We conjecture that there are almost always truth assignments with a core at all energy levels 
below any level at which a freezing transition can happen at given a. For example, this implies that 
one can pick an rj- value for a = 4.2, such that the freezing takes place at, say, E/N = 0.005 (Figure 
120(1 . Hence we believe that there exist solutions with a core also in the N — > 00 limit in some region 
of a- values, approximately 4.10 < a < a c . The lower bound of 4.10 has been estimated using FMS 
with rj = 0.01 (Figure l2*U)l . Since the choice of algorithm may have an effect on the estimate, the 
actual threshold point below which there do not exist solutions with a core in the iV — > 00 limit 
can be lower than this. 

There may be a narrow region below a c where typical instances do not have any completely 
white solutions. In this region, if it exists, focused algorithms presumably cannot find a solution 
in linear time. On the other hand, in [12| it is conjectured that almost all solutions are completely 
white ("all-* assignments" in the terminology of |l2l |) up to a c . This claim is based on experimental 
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FIG. 21: Sample runs of FMS at a = 4.15 (N — 10 5 ). Left: Metastable energies and freezing transition 
energies. Right: Solution times and freezing transition times. Dotted lines give a rough estimate of the 
(vii J7«)-window at this a. 
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FIG. 22: Solution times and freezing times, when using FMS with r\ = 0.05. Each point corresponds to a 
single run (iV = 10 5 ). 



15j], which seemingly finds only completely 



evidence using the survey propagation algorithm 4, 5, 

white solutions when N is large |l2(, just like focused local algorithms. However, if solutions with a 
core cannot be found in linear time, it is not surprising that usually the first solution encountered 
is completely white. Thus, the issue of a possible core transition (similar to the simpler XORSAT 
problem [FJ]) below a c remains open. We have not tried to assess which type of solution is more 
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FIG. 23: Proportion of "safe" clauses in solutions found by FMS. Each point corresponds to a single run 
(N = 10 6 ). Horizontal dotted line is this proportion, when all the satisfying combinations have equal 
probability: 4/7. Vertical dotted line is our estimate for the dynamical threshold of the Random Walk: 
aw 2.67. 

numerous near a c , or whether having a majority of completely white solutions is in fact a necessary 
or even a sufficient condition for maintaining time-linearity of local search methods. Thus, we 
can at the moment only conjecture a relation between the algorithmic performance of local search 
methods and the entropy of the solution space, as measured in terms of the "white" and "core" 
fractions vs. N. 

Another way to establish the existence of qualitatively different kinds of solutions is to look 
at the proportion of "safe" clauses (i.e. clauses that contain at least two satisfying literals) in the 
solutions found by FMS with different values of r/ (Figure EHJ) • One can say that with greater 77, 
better quality solutions are found. Note that these safe clauses are just what enable variables to 
become unfrozen in the whitening algorithm (Figure FT5]) . 

Whiteness is just one indicator that can be used during the search process to tell when the linear- 
time property has been lost. Certainly there can be more powerful indicators that alarm about 
difficulties earlier, when the freezing transition has not yet happened. If a truth assignment which 
is not a solution is completely white, it means that any one of the still remaining unsat clauses can 
be resolved locally (assuming dependency loops between clauses don't incidentally block this). It 
does not, however, mean that an arbitrary subset of the remaining unsat clauses could necessarily 
be resolved locally, or that there necessarily exists a solution. A truth assignment can turn from 
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FIG. 24: Proposed phase diagram for FMS. Above a dynamical threshold a metastable state is encountered 
before a solution is reached. Below a whitening threshold a freezing transition happens before a solution or 
a metastable or stable state is reached. The data points are rough estimates based on a few runs of the FMS 
algorithm. 

a completely white one to having a core by the flip of just one variable. One can envision higher 
order whitenings: a fcth order whitening would tell whether all truth assigments that differ in at 
most k variable values from the assignment in question are completely white (in the usual sense, 
i.e. zeroth order whitened). Maybe a higher order whitening with any fixed k can (in principle [3^) 
be used as an indicator for the linear-time property, just like the zeroth order one. There can also 
be some indicators that have nothing to do with whitening. [3^ 



VI. CONCLUSIONS 



In this paper we have elucidated the behaviour of "focused" local search algorithms for the 
random 3-SAT problem. An expected conclusion is that they can be tuned so as to extend the 
regime of "good", linear- in-N behaviour closer to a c . 

An unexpected conclusion is that FMS (and WalkSAT, and FRRT) seem to work well very close 
to the critical threshold. Figure l2*H proposes a phase diagram. FMS with rj = 1 is just the Random 
Walk algorithm; hence the first transition point is at a ~ 2.67. For larger a there is the possibility 
of having two phase boundaries in terms of the noise parameter. The upper value r) u separates the 
linear regime from one with too much noise, in which the fluctuations of the algorithm degrade its 



performance. For rj < rji, greediness leads to dynamical freezing, and though FMS (in contrast ?e 
FRRT in particular) remains ergodic, or able to climb out of local minima, the algorithm no longer 
scales linearly in the number of variables. 

The phase diagram presents us with two main questions: what is the smallest a at which 
rji(a) starts to deviate from zero? Does the choice of an ideal rj op t allow one to push the linear 
regime arbitrarily close to a c ? Note that the deviation of rj u (a) from unity with increasing a could 
perhaps be analysed with similar techniques as the methods in refs. P, l^jj. The essential idea 
there is to construct rate equations for densities of variables or clauses while ignoring correlations 
of neighboring ones. This would not resolve the above two main ones, of course. 

The resolution of these questions will depend on our understanding the performance of FMS or 
other local search presence of the clustering of solutions. The whitening or freezing experiments 
provide qualitative understanding on the behaviour for small rj: too greedy search procedures are 
not a good idea. Thus FMS seems to work well only when it actually finds solutions that can be 
completely whitened. Figure l2*H also points out, via the whitening data for a > a c , that the line 
of whitening thresholds rji for FMS apparently is continuous over a c . In particular in this case, 
one can then separate between two possible scenarios: either the r] u -lme extends to a c and finishes 
there, or otherwise it meets with the r//-line at an a < a c at a point that is "tri-critical" in statistical 
mechanics terms. Note, however, that rji determined by whitening is not necessarily the highest 
lower bound to the linear phase. 

The range of the algorithm presents therefore us with a fundamental question: though replica- 
methods have revealed the presence of clustering in the solution space starting from a ~ 3.92, 
the FMS works for much higher a's still. The clustered solutions should have an extensive core 
of frozen variables, and therefore to be hard to find. Thus, the existence of "easy solutions" for 
the FMS tells that there are features in the solution space and energy landscape that are not well 
understood, since the FMS can pick, for a right choice of rj almost always such easy solutions out of 
those available for any particular instance. This would seem to imply that such completely white 
ones have a considerable entropy. 

Our numerics also implies that in the linear scaling regime the solution time probability distri- 
butions become sharper ("concentrate") as N increases, implying indeed that the median solution 
time scales linearly. We can by numerical experiments not prove that this holds also for the aver- 
age behaviour, but would like to note that the empirically observed tail behaviours for the same 
distributions give no indication of such heavy tails that would contradict this idea. 
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